Experience vs. description based decision-making project: Raw behavioral findings

Set up

clean_beh_data = clean_beh_data %>%
  mutate(EVRight = referenceProb * referenceValue,
         EVLeft = lotteryValue * lotteryProb,
         conflictTrial = ifelse( (EVLeft<EVRight) & (fractalLeftProb>fractalRightProb), "conflict", ifelse( (EVLeft>EVRight) & (fractalLeftProb<fractalRightProb), "conflict", "no conflict")), #this puts all EVLeft == EVRight trials into the no conflict bin
         leftLotteryBetter = EVLeft-EVRight>0,
         leftFractalBetter = (fractalLeftProb)-(fractalRightProb)>0,
         lotteriesSame = EVLeft == EVRight,
         fractal_diff = abs(fractalLeftProb - fractalRightProb),
         lottery_ev_diff = round(abs(EVLeft - EVRight),3),
         lottery_ev_diff = ifelse(lottery_ev_diff ==0, "no EV diff", ifelse(lottery_ev_diff == .2, "small EV diff", ifelse(lottery_ev_diff == .4, "large EV diff", NA))),
         lottery_ev_diff = factor(lottery_ev_diff, levels=c("no EV diff", "small EV diff", "large EV diff"), labels = c("no", "small", "large")),
         choseBetterLotteryBundle = ifelse(leftLotteryBetter & choiceLeft == 1, 1, ifelse(!leftLotteryBetter & choiceLeft == 0, 1, 0)),
         choseBetterFractalBundle = ifelse(leftFractalBetter & choiceLeft == 1, 1, ifelse(!leftFractalBetter & choiceLeft == 0, 1, 0)),
         leftBundleValue = (probFractalDraw*fractalLeftProb) + ((1-probFractalDraw)*EVLeft),
         rightBundleValue = (probFractalDraw * fractalRightProb) + ((1-probFractalDraw)*EVRight),
         leftBundleBetter = leftBundleValue > rightBundleValue,
         choseBetterOverallBundle = ifelse(leftBundleBetter & choiceLeft == 1, 1, ifelse(!leftBundleBetter & choiceLeft == 0, 1, 0)),
         logRt = log(reactionTime),
         probFractalDraw = as.factor(probFractalDraw))

fig_out_path = paste0(here(), '/outputs/fig/')

Choice

Analyzing choice without using any model-based measures

Side

Proportion of left choice by probFractalDraw for all subjects. No systematic idiosyncratic side bias.

p = clean_beh_data %>%
  group_by(subnum, probFractalDraw, conflictTrial) %>%
  summarise(.groups = "keep",
            propLeft = sum(choiceLeft)/n()) %>%
  ggplot(aes(probFractalDraw, propLeft))+
  geom_boxplot()+
  # geom_jitter(height=0, width=.1, alpha=.5, aes(color=as.numeric(subnum)))+
  geom_hline(yintercept=.5, color="gray")+
  theme(legend.position = "none")+
  facet_grid(conflictTrial~.)+
  labs(x="p(Fractal)", y="Proportion of left choice")

# ggsave(file=paste0(fig_out_path, 'prop_left_choice.jpg'), p, height = 5, width=8, units="in")
p

Choice based on lottery, fractal, bundle value

When the lotteries don’t have the same EV do subjects choose the bundle with the better lottery? This should depend on how much the reward depends on the lotteries. Subject do choose the bundle with the better lottery more frequently when it matters for the reward. They appropriately choose it less frequently when the reward depends more on the fractals and the fractals suggest a different bundle to be the better one (conflict trials)

Do they get better with time?
- Some improvement for pFracDraw <=.5 especially from run 1 to run 2
- Even for pFracDraw <= .5 trials no conflict trials are slightly worse than conflict trials.
- For trials where fractals matter more for the reward (pFracDraw > .5) performance is worse in both types of conflict trials. In conflict trials the drop for choosing the bundle with the better lottery is not as sharp as it should be and in no conflict trials choice of bundle with better lottery is not as high even though it is also the bundle with the better fractal.

p = clean_beh_data %>%
  mutate(session = paste0("run ", session),
         choseBetterLotteryBundle = ifelse(EVRight == EVLeft, NA, choseBetterLotteryBundle),
         choseBetterOverallBundle = ifelse(leftBundleValue == rightBundleValue, NA, choseBetterOverallBundle)) %>%
  group_by(probFractalDraw, conflictTrial, session) %>%
  summarise(.groups="keep",
            prop_choseBetterLotteryBundle = mean(choseBetterLotteryBundle, na.rm=T),
            sem_choseBetterLotteryBundle = sd(choseBetterLotteryBundle, na.rm=T)/sqrt(n()),
            prop_choseBetterFractalBundle = mean(choseBetterFractalBundle),
            sem_choseBetterFractalBundle = sd(choseBetterFractalBundle)/sqrt(n()),
            prop_choseBetterOverallBundle = mean(choseBetterOverallBundle, na.rm=T),
            sem_choseBetterOverallBundle = sd(choseBetterOverallBundle, na.rm=T)/sqrt(n())) %>%
    gather(prop_type, prop_value, -probFractalDraw, -conflictTrial, -session,-sem_choseBetterLotteryBundle ,-sem_choseBetterFractalBundle, -sem_choseBetterOverallBundle) %>%
  gather(sem_type, sem_value,  -probFractalDraw, -conflictTrial, -session, -prop_type, -prop_value) %>%
  ggplot(aes(probFractalDraw, prop_value, color=prop_type))+
  geom_point(alpha=.5)+
  geom_errorbar(aes(ymin = prop_value-sem_value,
                    ymax = prop_value+sem_value), width=.2)+
  geom_hline(aes(yintercept=.5), color="grey")+
  facet_grid(conflictTrial~session)+
  theme(legend.position = "bottom")+
  labs(color="", x="p(Fractal)", y="Proportion of choices")+
  scale_color_manual(values=c(cbbPalette[1:2], "purple"),
                     breaks = c("prop_choseBetterFractalBundle", "prop_choseBetterLotteryBundle", "prop_choseBetterOverallBundle"),
                     labels = c(c("Better Fractal Bundle", "Better Lottery Bundle", "Better Overall Bundle")))+
  scale_x_discrete(breaks = c("0", "0.2", "0.4", "0.6", "0.8", "1"))+
  ylim(0,1)

# ggsave(file=paste0(fig_out_path, 'prop_better_choice.jpg'), p, height = 5, width=10, units="in")
p

Logit analysis with true values

p = clean_beh_data %>%
  select(EVLeft, EVRight, fractalLeftProb, fractalRightProb, probFractalDraw, choiceLeft) %>%
  mutate(EVDiff = scale(EVLeft - EVRight), 
         fractalDiff = scale(fractalLeftProb - fractalRightProb)) %>%
  nest(data = -probFractalDraw) %>% 
  mutate(
    fit = map(data, ~ glm(choiceLeft ~ EVDiff + fractalDiff, data = .x, family=binomial(link="logit"))),
    tidied = map(fit, tidy)
  ) %>% 
  unnest(tidied) %>%
  filter(term != "(Intercept)") %>%
  select(probFractalDraw, term, estimate, std.error) %>%
  mutate(term = ifelse(term == "EVDiff", "Lottery EV difference", ifelse(term == "fractalDiff", "Fractal probability difference", NA))) %>%
  ggplot(aes(probFractalDraw, estimate, col=term, group=term))+
      geom_point()+
      geom_line()+
      geom_errorbar(aes(ymin = estimate - std.error, ymax = estimate +std.error), width=0.2)+
      geom_hline(aes(yintercept=0), linetype="dashed")+
      scale_color_manual(values = cbbPalette[1:2])+
      theme(legend.position = "bottom")+
      labs(color="", y="Logit slope estimate", x="p(Fractal)")

# ggsave(file=paste0(fig_out_path, 'group_logit.jpg'), p, height = 5, width=8, units="in")
p

Ambiguity aversion

Can a single constant ambiguity aversion parameter explain the discrepancy between the logit slopes for EV and fractal prob differences?

Is the ratio of the slopes for EV at (1-p) and QV at p constant?

clean_beh_data %>%
  select(EVLeft, EVRight, fractalLeftProb, fractalRightProb, probFractalDraw, choiceLeft) %>%
  mutate(EVDiff = scale(EVLeft - EVRight), 
         fractalDiff = scale(fractalLeftProb - fractalRightProb)) %>%
  nest(data = -probFractalDraw) %>% 
  mutate(
    fit = map(data, ~ glm(choiceLeft ~ EVDiff + fractalDiff, data = .x, family=binomial(link="logit"))),
    tidied = map(fit, tidy)
  ) %>% 
  unnest(tidied) %>%
  filter(term != "(Intercept)") %>%
  select(probFractalDraw, term, estimate, std.error) %>%
  mutate(probFractalDraw = as.numeric(as.character(probFractalDraw)),
         attributeRelevance = ifelse(term == "EVDiff", 1-probFractalDraw, probFractalDraw),
         attributeRelevance = as.factor(attributeRelevance),
         term = ifelse(term == "EVDiff", "Lottery EV difference", ifelse(term == "fractalDiff", "Fractal probability difference", NA))) %>%
  ggplot(aes(attributeRelevance, estimate, col=term, group=term))+
      geom_point()+
      geom_line()+
      geom_errorbar(aes(ymin = estimate - std.error, ymax = estimate +std.error), width=0.2)+
      geom_hline(aes(yintercept=0), linetype="dashed")+
      scale_color_manual(values = cbbPalette[1:2])+
      theme(legend.position = "bottom")+
      labs(color="", y="Logit slope estimate", x="Attribute relevance")

When fractal matters equally or more for the reward the ratio of EV slope to QV slope, i.e. ambiguity aversion relatively constant.

Initially I added error bars to this plot are added using the Taylor expansion for the second moment (variance) of a quotient as described here extracting the relevant statistics from the variance covariance matrix of the regression. But they didn’t look sensible because they were much smaller than the error bars of the coefficient estimates above.

So this plot is more for demonstration purposes and the idea of whether a constant ambiguity aversion can account for the discrepancy between the effects of each attribute on choice is tested more systematically in NB17_DDRLLinearDistortion.

tmp = clean_beh_data %>%
  select(EVLeft, EVRight, fractalLeftProb, fractalRightProb, probFractalDraw, choiceLeft) %>%
  mutate(EVDiff = scale(EVLeft - EVRight), 
         fractalDiff = scale(fractalLeftProb - fractalRightProb)) %>%
  nest(data = -probFractalDraw) %>% 
  mutate(
    fit = map(data, ~ glm(choiceLeft ~ EVDiff + fractalDiff, data = .x, family=binomial(link="logit"))),
    tidied = map(fit, tidy)
  ) %>% 
  unnest(tidied) %>%
  filter(term != "(Intercept)") %>%
  mutate(varEVDiff = NA, varFracDiff = NA, covEVFracDiff = NA, n=NA)


tmp %>%
  select(probFractalDraw, n, term, estimate, varEVDiff, varFracDiff, covEVFracDiff) %>%
  spread(term, estimate) %>%
  mutate(ambAve = rev(EVDiff)/fractalDiff) %>%
  ggplot(aes(probFractalDraw, ambAve))+
  geom_point()+
  geom_line(group=1, color="gray")+
  geom_hline(aes(yintercept=1), linetype="dashed")+
  annotate("text", x = 10, y = 1.25, label = "ambiguity neutral")+
  labs(x = "Attribute relevance", y="slope(EVDiff)/slope(fractalDiff)\nat equivalent relevance level")

Perseverance of choice

When reward depends more on fractals and the previous trial was rewarded based on the fractals then subjects are more likely to choose the side they chose previously. Makes sense. Suggestive of learning and understanding of task.

p = clean_beh_data %>%
  select(subnum, trialNum, choiceLeft, reward, fractalDraw, probFractalDraw) %>%
  group_by(subnum) %>% # so the last choice for one subject does not bleed into the first choice of the next subject
  mutate(lastChoiceSide = lag(choiceLeft),
         lastFractalDraw = lag(fractalDraw),
         choseLastSide = choiceLeft == lastChoiceSide,
         lastTrialRewarded = paste0("Last Trial rewarded: ", lag(reward > 0))) %>%
  filter(trialNum > 1) %>% #remove first trial of each run
  drop_na() %>% #for two subjects there are cases when trialNum == 1 is cleaned because there was no response. These lead to NAs in lastChoiceSide and lastFractalDraw so they are dropped
  ungroup()%>%
  group_by(lastFractalDraw, lastTrialRewarded, probFractalDraw) %>%
  summarise(.groups='keep',
            prop_choseLastSide = mean(choseLastSide, na.rm=T),
            sem_choseLastSide = sd(choseLastSide, na.rm=T)/sqrt(n())) %>%
  mutate(lastRewardAttribute = ifelse(lastFractalDraw == 1, "fractal", "lottery")) %>%  
  ggplot(aes(probFractalDraw, prop_choseLastSide, color=lastRewardAttribute))+
  geom_point(position = position_dodge(width=.5))+
  geom_errorbar(aes(ymin=prop_choseLastSide-sem_choseLastSide, ymax=prop_choseLastSide+sem_choseLastSide), width=.2, position=position_dodge(width=.5))+
  geom_hline(aes(yintercept=.5), color="gray")+
  facet_grid(lastTrialRewarded~.)+
  theme(legend.position = "bottom")+
  labs(x= "p(Fractal)", y = "Proportion of repeated choice", color="Last rewarded attribute")+
  scale_color_manual(values=cbbPalette[1:2],
                     breaks = c("fractal", "lottery"),
                     labels = c("Fractal", "Lottery"))

# ggsave(file=paste0(fig_out_path, 'perseverance_of_choice.jpg'), p, height = 5, width=8, units="in")
p

Learning of fractal values

Fractal reward probabilities

Is there a good signal to learn? Learn what? Learn which fractal is better from the other? Whether a fractal is good vs bad? These two are not the same. A signal of which fractal is better depends on the difference between the two fractal probabilities. A signal for whether a fractal is good or bad depends on how far the fractal probability is from chance.

Neither of these signals are constant for any subject across runs. So in each run a subject might learn a different thing about the fractals.

clean_beh_data %>%
  group_by(subnum) %>%
  mutate(trialNum = 1:n()) %>%
  select(fractalLeftProb, fractalRightProb, trialNum, subnum) %>%
  gather(key, value, -trialNum, -subnum) %>%
  ggplot()+
  geom_line(aes(trialNum, value, color=key))+
  geom_vline(xintercept=60, color="gray")+
  geom_vline(xintercept=120, color="gray")+
  geom_vline(xintercept=180, color="gray")+
  geom_vline(xintercept=240, color="gray")+
  # geom_point(data=tmp, aes(x=trialNum, y=estimate*10))+
  # geom_errorbar(data = tmp, aes(ymin=10*estimate-std.error, ymax=10*estimate+std.error), width=.2)+
  # facet_wrap(~subnum, scales="free_y")+
  facet_wrap(~subnum)+
  theme(legend.position = "bottom",
        legend.title=element_blank())

To check this I looked at whether subjects are more likely to choose the fractal with the better bundle later in a run for each run. Note that these are pretty underpowered models since they’re run on 60 trials each and for a large chunk of each run fractals aren’t even the more relevant attribute for reward (filtering the trials where the fractals were at least as relevan is even more underpowered and doesn’t lead to a different conclusion).

These slopes are not significantly different from 0 most of the time, i.e. subjects are not more likely to choose the bundle with the better fractal later in a run. The pattern between the fractal probabilities is not consistent for cases where the slopes are different from 0. So this perspective did not provide a clear raw behavioral measure of learning.

tmp = clean_beh_data %>%
  # filter(as.numeric(as.character(probFractalDraw))>.5) %>%
  select(subnum, session, trialNum, choseBetterFractalBundle) %>%
  nest(data = -c(subnum, session)) %>% 
  mutate(
    fit = map(data, ~ glm(choseBetterFractalBundle ~ trialNum, data = .x, family=binomial(link="logit"))),
    tidied = map(fit, tidy)
  ) %>% 
  unnest(tidied) %>%
  filter(term != "(Intercept)") %>%
  select(subnum, session, estimate, std.error) %>%
  mutate(trialNum = 30*((session*2)-1),
         session = paste0("run ", session))

p = tmp %>%
  ggplot(aes(session, estimate))+
  geom_point()+
  geom_errorbar(aes(ymin=estimate-std.error, ymax=estimate+std.error), width=.2)+
  geom_hline(aes(yintercept=0), color="gray")+
  facet_wrap(~subnum, scale="free_y")+
  xlab("")+
  ylab("")+
  theme(panel.grid = element_blank(),
        panel.background = element_rect(fill='transparent'),
         plot.background = element_rect(fill='transparent', color=NA),
        axis.ticks = element_blank(),
        axis.text = element_blank())+
  scale_y_continuous(position = "right")

# ggsave(file=paste0(fig_out_path, 'betterFractalSlopes.png'), p, height = 5, width=8, units="in", bg='transparent')
p

# Comparison figure on slide
p =clean_beh_data %>%
  group_by(subnum) %>%
  mutate(trialNum = 1:n()) %>%
  select(fractalLeftProb, fractalRightProb, trialNum, subnum) %>%
  gather(key, value, -trialNum, -subnum) %>%
  ggplot()+
  geom_line(aes(trialNum, value, color=key))+
  geom_vline(xintercept=60, color="gray")+
  geom_vline(xintercept=120, color="gray")+
  geom_vline(xintercept=180, color="gray")+
  geom_vline(xintercept=240, color="gray")+
  facet_wrap(~subnum, scale="free_y")+
  theme(legend.position = "none",
        legend.title=element_blank(),
        # panel.spacing = unit(1, "lines"),
        panel.grid = element_blank(),
        axis.text = element_blank(),
        axis.ticks = element_blank())+
  xlab("")+
  ylab("")

# ggsave(file=paste0(fig_out_path, 'fractalProbs.png'), p, height = 5, width=8, units="in", bg='transparent')
# p

Looking at things at the group level for more power. Are you more likely to choose the better fractal bundle as trialnum increases? No. No difference depending on conflict trial either.

clean_beh_data %>%
  mutate(session = paste0("run ", session)) %>%
  ggplot(aes(trialNum, choseBetterFractalBundle))+
  geom_smooth(formula = 'y~x', method = "glm", method.args = list(family=binomial), fullrange=TRUE, color=cbbPalette[3], alpha=.75)+
  # geom_line(aes(group=subnum),stat='smooth',formula = 'y~x', method = "lm",se=FALSE, alpha=.1, size=1)+
  facet_grid(conflictTrial~session)+
  labs(x = "Trial Number", y="Choice of fractal with higher objective reward probability")

Are you more likely to choose the better fractal bundle as trialnum increases for trials where the fractals matter more for the reward? Somewhat in earlier runs. THis again is weak and not consistent. Plus, it would need an explanation for why in the last two runs the effect disappears.

clean_beh_data %>%
  filter(as.numeric(as.character(probFractalDraw))>.5) %>%
  mutate(session = paste0("run ", session)) %>%
  ggplot(aes(trialNum, choseBetterFractalBundle))+
  geom_smooth(formula = 'y~x', method = "glm", method.args = list(family=binomial), fullrange=TRUE, color=cbbPalette[3], alpha=.75)+
  # geom_line(aes(group=subnum),stat='smooth',formula = 'y~x', method = "lm",se=FALSE, alpha=.1, size=1)+
  facet_grid(conflictTrial~session)+
  labs(x = "Trial Number", y="Choice of fractal with higher objective reward probability", title="For p(Fractal)>.5")

Is a group level learning effect masked due to individual differences? No, the within subject slopes are not consistent across runs.

Are you more likely/above chance to choose the bundle with the better fractal in the second half of each run when fractals matter more? Looked at this for the whole runs in the first large plot in the “Choice based on lottery, fractal, bundle value” section. It did not suggest strong learning signals for the conflict trials (looks above chance for no conflict but you can’t tell in these trials if it’s because the lottery is better).

Maybe choice of objectively better bundle is not revealing learning because subjects don’t know this but only know how frequently a fractal is rewarded.

Are they more likely to choose the bundle with fractal that was rewarded more frequently in the last 4 trials weighted by its relevance in the given trial? (something like a good/bad fractal signal that decays with time and is weighted by its relevance in each trial). This is a very convoluted measure and I’m not sure it is indicative of anything useful. So far it hasn’t looked like learning is happening well and this doesn’t suggest anything different, if anything at all.

clean_beh_data %>%
  group_by(subnum, session) %>%
  mutate(decayingCumSumNumLeftFractalRewarded  = zoo::rollapplyr(1:n(),
                                                   width = 5,
                                                   FUN = function(ix) weighted.mean(leftFractalReward[ix], c(.1, .2, .3, .4, 0)),
                                                   fill = NA),
         decayingCumSumNumRightFractalRewarded = zoo::rollapplyr(1:n(),
                                                   width = 5,
                                                   FUN = function(ix) weighted.mean(rightFractalReward[ix], c(.1, .2, .3, .4, 0)),
                                                   fill = NA),
         probFractalDraw = as.numeric(as.character(probFractalDraw)),
         weightedNumLeftFractalRewarded = probFractalDraw * decayingCumSumNumLeftFractalRewarded,
         weightedNumRightFractalRewarded = probFractalDraw * decayingCumSumNumRightFractalRewarded,
         choseMoreRewardedFractal = ifelse((weightedNumLeftFractalRewarded> weightedNumRightFractalRewarded) & (choiceLeft==1), 1, ifelse((weightedNumRightFractalRewarded>weightedNumLeftFractalRewarded) & (choiceLeft==0), 1, 0))) %>%
  # select(probFractalDraw, leftFractalReward, rightFractalReward, weightedNumLeftFractalRewarded, weightedNumRightFractalRewarded)
  ungroup() %>%
  mutate(session = paste0("run ", session)) %>%
  filter(trialNum>5) %>%
  ggplot(aes(trialNum, choseMoreRewardedFractal))+
  geom_smooth(formula = 'y~x', method = "glm", method.args = list(family=binomial), fullrange=TRUE, color=cbbPalette[3], alpha=.75)+
  # geom_line(aes(group=subnum),stat='smooth',formula = 'y~x', method = "glm",se=FALSE, alpha=.1, size=1)+
  facet_grid(.~session)+
  labs(x = "Trial Number", y="Choice of more frequently rewarded fractal\n in the last five trials")

## Warning: Removed 1 rows containing non-finite values (stat_smooth).

Overall I have not found a clear signal for learning in raw behavioral data.

One suggestion from lab meeting was to compare predicted probability versus true proportion of choice for different levels of attribute relevance (?). I’ve done this before by sampling from subject posteriors n=100 times, generating choice data and comparing it true subject data. Overall we found that most subjects’ choice at various levels fell in the middle of the posterior predictive distribution. This suggests that the choice behavior in the task can be recapitulated by RL models to a decent degree. But the subject facetted way of presenting this was difficult to digest.

Here is a more succinct way of visualizing and summarizing posterior predictive data/model fit. Each dot represents a subject and each facet a fractal relevance level. We see that there are differences both across subjects and across relevance levels but overall the model can generate reasonable looking data.

Note, however, the sequential nature of the model is crucial and this summary does not capture that. Such a sequential change in raw behavior was what I was looking for in raw behavior but I couldn’t find one.

modelName = 'fit_rl_hierarchical_oneParamProbDistortion'

if(file.exists(paste0(helpers_path, 'rlModels/pp_data_', modelName, '.RDS'))){
  pp_data = readRDS(paste0(helpers_path, 'rlModels/pp_data_', modelName, '.RDS'))
} else{
  pp_data = make_posterior_predictive_data(numDraws = 100, modelName = modelName) 
}

tmp = clean_beh_data %>%
  group_by(subnum, probFractalDraw) %>%
  summarise(true_propLeft = mean(choiceLeft), .groups='keep')

pp_data %>%
  group_by(subnum, sampleNum, probFractalDraw) %>%
  summarise(propLeft = mean(choiceLeft), .groups='keep') %>%
  mutate(data_type = "predicted") %>%
  group_by(subnum, probFractalDraw) %>%
  summarise(.groups="keep",
            mean_propLeft = mean(propLeft),
            sem_propLeft = sd(propLeft)/sqrt(n())) %>%
  mutate(subnum = as.factor(subnum),
         probFractalDraw = as.factor(probFractalDraw)) %>%
  left_join(tmp, by=c("subnum","probFractalDraw")) %>%
  ggplot(aes(x=true_propLeft, y=mean_propLeft, color=subnum))+
  # geom_point()+
  geom_errorbar(aes(ymin=mean_propLeft-sem_propLeft, ymax=mean_propLeft+sem_propLeft), width=0.05)+
  geom_abline(aes(intercept=0, slope=1), color="gray")+
  facet_wrap(~probFractalDraw)+
  theme(panel.grid = element_blank(),
        axis.text = element_blank(),
        axis.ticks = element_blank(),
        legend.position="none")+
  xlab("True proportion of left choice")+
  ylab("Predicted proportion of left choice\nfrom 100 posterior samples")

Differential learning

Is there:
- more learning for rewarded side?
- more chosen bundle fractal?
- more learning from rare events? E.g. if the fractaldraw wasn’t likely but happened anyway?

Because I don’t have a clear signal for learning in raw behavior I’m not sure how to test these ideas in behavior alone. Instead, I will expand on the analyses I did here fitting RL models with different update rules and checking whether they explain data better.

Risk attitude

Risk attitude is a focus when studying the description-experience gap.
The finding is that in description-based choice subjects are risk averse in gains and risk seeking in losses and in experience-based choice this flips, such that subjects are risk seeking in gains and risk averse in losses.

To make contact with this literature anticipating any reviewer comments on this we considered assuming an exponential utility function \(u(c) = 1-e^{(-\alpha c)}\) which exhibits constant absolute risk aversion \(\alpha\) and test if this parameter is different for the extreme of attribute relevance when the choice is (/should be) either experience- or description-based.

I decided not to go through with this because there are several issues with this approach:
- It would be estimated on too few trials
- Assumes a lot in functional forms especially since in neither case there is an explicit certainty equivalent
- Description-based choice is not description only since it’s outcome is played out in each trial

Choice patterns to model

• Better performance for trials where the lotteries matter more for the reward. When fractal matters more choice is around chance in conflict trials and above chance but worse than lottery mattering trials for no conflict trials.
  
• Asymmetric distortion of attribute relevance: Lottery slopes decline linearly but fractal slopes might have a more step-like shape. The step-like is not confirmed but there is definitely an underweighting of the fractal value difference compared the lottery value difference.
  
• When reward depends more on fractals and the previous trial was rewarded based on the fractals then subjects are more likely to choose the side they chose previously.
  
• Did not find a clean signal of learning/sequential improvement of behavior from raw behavioral measures so it’s difficult to test specific hypotheses of differential learning.

Reaction Time

Attribute relevance

Decisions are slowest when both attributes are equally relevant. They are also categorically faster when only fractals matter for reward.

p = clean_beh_data %>%
  group_by(probFractalDraw) %>%
  summarise(meanLogRt = mean(logRt),
            semLogRt = sd(logRt)/sqrt(n())) %>%
  ggplot(aes(probFractalDraw, meanLogRt))+
  geom_point()+
  geom_errorbar(aes(ymin=meanLogRt - semLogRt, ymax=meanLogRt + semLogRt), width=.2)+
  labs(y="mean log RT", x="p(Fractal)")

ggsave(file=paste0(fig_out_path, 'rt_by_probfractaldraw.jpg'), p, height = 5, width=8, units="in")
p

Do we see this pattern for all subjects? Most but not all. Subjects 9 nd 10 look particularly weird.

clean_beh_data %>%
  group_by(subnum, probFractalDraw) %>%
  summarise(.groups="keep",
            meanLogRt = mean(logRt),
            semLogRt = sd(logRt)/sqrt(n())) %>%
  ggplot(aes(probFractalDraw, meanLogRt))+
  geom_point()+
  geom_errorbar(aes(ymin=meanLogRt - semLogRt, ymax=meanLogRt + semLogRt), width=.2)+
  facet_wrap(~subnum, scales="free_y")

Does RT depend on any other experimental manipulation?

True fractal values

Does it depend on the objective fractal values? Not on average but there are individual differences.

clean_beh_data %>%
  group_by(subnum) %>%
  mutate(fractal_prob_diff = ifelse(fractal_diff < quantile(fractal_diff, probs=c(.33))[[1]], "small",
                                    ifelse(fractal_diff > quantile(fractal_diff, probs=c(.66))[[1]], "large", "medium")),
         fractal_prob_diff = factor(fractal_prob_diff, levels=c("small", "medium", "large"))) %>%
  ungroup() %>%
  group_by(probFractalDraw, fractal_prob_diff) %>%
  summarise(.groups = "keep",
            mean_logRt = mean(logRt),
            sem_logRt = sd(logRt)/sqrt(n())) %>%
  ggplot(aes(probFractalDraw, mean_logRt,color=fractal_prob_diff))+
  geom_point(position=position_dodge(width=.5))+
  geom_errorbar(aes(ymin = mean_logRt - sem_logRt, ymax = mean_logRt + sem_logRt), width=0,position=position_dodge(width=.5))+
  labs(color="Fractal prob difference", y="Mean Log RT", x="p(Fractal)")+
  scale_color_manual(values = c(cbbPalette[3], cbbPalette[5:6]))+
   theme(legend.position = "bottom")

EV of varying lottery

Does it depend on the amount or probability of the varying lottery? No. Not for anyone (or any probFractalDraw level; not shown).

clean_beh_data %>%
  select(subnum, reactionTime, lotteryValue, lotteryProb) %>%
  gather(key, value, -subnum, -reactionTime) %>%
  ggplot(aes(value, reactionTime))+
  geom_line(aes(group=subnum),stat='smooth',formula = 'y~x', method = "lm",se=FALSE, alpha=.1, size=1)+
  geom_smooth(formula = 'y~x', method = "lm")+
  facet_wrap(~key, scales="free_x")

Conflict trials

Does it depend on whether the objective values for each attribute agree or disagree on which is the better bundle? No.

clean_beh_data %>%
  group_by(probFractalDraw, conflictTrial) %>%
  summarise(.groups="keep",
            meanLogRt = mean(logRt),
            semLogRt = sd(logRt)/sqrt(n())) %>%
  ggplot(aes(probFractalDraw, meanLogRt, color=conflictTrial))+
  geom_point()+
  geom_errorbar(aes(ymin=meanLogRt - semLogRt, ymax=meanLogRt + semLogRt), width=.2)+
  theme(legend.position = "bottom",
        legend.title = element_blank())

Lottery EV difference

Choice is faster for larger differences between the lotteries when lotteries are more relevant for reward. Otherwise the effect disappears.

p = clean_beh_data %>%
  group_by(probFractalDraw, lottery_ev_diff) %>%
  summarise(.groups = "keep",
            mean_logRt = mean(logRt),
            sem_logRt = sd(logRt)/sqrt(n())) %>%
  ggplot(aes(probFractalDraw, mean_logRt,color=lottery_ev_diff))+
  geom_point(position=position_dodge(width=.5))+
  geom_errorbar(aes(ymin = mean_logRt - sem_logRt, ymax = mean_logRt + sem_logRt), width=.2,position=position_dodge(width=.5))+
  theme(legend.position = "bottom")+
  labs(color="Lottery EV difference", y="Mean Log RT", x="p(Fractal)")+
  scale_color_manual(values = c(cbbPalette[3], cbbPalette[5:6]))

# ggsave(file=paste0(fig_out_path, 'rt_by_lottery_ev_diff.jpg'), p, height = 5, width=8, units="in")
p

Larger fractal value differences does not translate to faster choice. There are large individual differences but the average effects are either 0 or in the opposite direction.

Why not?
Because subjects are not aware/have not learned the value difference?
Because information about the fractals has been processed before the stimulus presentation so the whatever effect fractal value difference might have had is dissipated when we begin measuring RT and the only stimulus processing that happens from that point on is about the lotteries and integration of bundle values?

Learning

Does it depend on trial number/learning how to do the task? Yes. Subjects get faster in later runs on average. In the first run they also get faster with each trial.

p = clean_beh_data %>%
  mutate(session = paste0("run ", session)) %>%
  ggplot(aes(trialNum, logRt))+
  geom_line(aes(group=subnum),stat='smooth',formula = 'y~x', method = "lm",se=FALSE, alpha=.1, size=1)+
  geom_smooth(formula = 'y~x', method = "lm")+
  facet_grid(.~session)+
  labs(x = "Trial Number", y = "log RT")
  
  
# ggsave(file=paste0(fig_out_path, 'rt_by_session.jpg'), p, height = 5, width=8, units="in")
p

Choice side

Are they faster when they choose left vs right? Since the learned attribute options are both fractals the stimulus difference lies in the lotteries. The bundle on the right has the fixed and the one on the right has the varying lottery. So left choices might be slower since they require more processing.

This doesn’t seem to be the case overall.

clean_beh_data %>%
  group_by(choiceLeft, probFractalDraw) %>%
  summarise(.groups="keep",
            mean_logRt = mean(logRt),
            sem_logRt = sd(logRt)/sqrt(n())) %>%
  mutate(choiceLeft = ifelse(choiceLeft == 0, "right (varying Lottery)", "left (reference Lottery)")) %>%
  ggplot(aes(probFractalDraw, mean_logRt, color=choiceLeft))+
  geom_point()+
  geom_errorbar(aes(ymin = mean_logRt - sem_logRt, ymax = mean_logRt + sem_logRt), width=.2)+
  theme(legend.position = "bottom",
        legend.title = element_blank())

There isn’t a difference between the lack of this effect depending on whether there is a conflict between the attributes.

clean_beh_data %>%
  group_by(choiceLeft, probFractalDraw, conflictTrial) %>%
  summarise(.groups="keep",
            mean_logRt = mean(logRt),
            sem_logRt = sd(logRt)/sqrt(n())) %>%
  mutate(choiceLeft = ifelse(choiceLeft == 0, "right (varying Lottery)", "left (reference Lottery)")) %>%
  ggplot(aes(probFractalDraw, mean_logRt, color=choiceLeft))+
  geom_point()+
  geom_errorbar(aes(ymin = mean_logRt - sem_logRt, ymax = mean_logRt + sem_logRt), width=.2)+
  theme(legend.position = "bottom",
        legend.title = element_blank())+
  facet_grid(conflictTrial~.)

Choice based on lottery

When the lotteries don’t have the same EV (80% of the trials) are subjects faster when they choose the bundle with the better lottery? Yes, when the lottery matters more for the reward. This suggests less/no integration of additional attribute (fractals) when correctly choosing based on the relevant attribute. Trials where the lotteries mattered more for the reward but the better lottery bundle was not selected could be considered “errors”.

p = clean_beh_data %>%
  filter(!lotteriesSame) %>%
  group_by(probFractalDraw, choseBetterLotteryBundle) %>%
  summarise(.groups="keep",
            mean_logRt = mean(logRt),
            sem_logRt = sd(logRt)/sqrt(n())) %>%
  mutate(choseBetterLotteryBundle = ifelse(choseBetterLotteryBundle ==1, "TRUE", "FALSE")) %>%
  ggplot(aes(probFractalDraw, mean_logRt, color=choseBetterLotteryBundle))+
  geom_point()+
  geom_errorbar(aes(ymin = mean_logRt - sem_logRt, ymax = mean_logRt + sem_logRt), width=.2)+
  theme(legend.position = "bottom")+
  labs(x="p(Fractal)", color="Chose Bundle with Better Lottery", y = "mean log RT")+
  scale_color_manual(values = cbbPalette[1:2])

# ggsave(file=paste0(fig_out_path, 'rt_by_correct_lottery.jpg'), p, height = 5, width=8, units="in")
p

The effect is driven by different types of conflict trials at different levels of probFractalDraw.

clean_beh_data %>%
  filter(!lotteriesSame) %>%
  group_by(probFractalDraw, choseBetterLotteryBundle, conflictTrial) %>%
  summarise(.groups="keep",
            mean_logRt = mean(logRt),
            sem_logRt = sd(logRt)/sqrt(n())) %>%
  mutate(choseBetterLotteryBundle = ifelse(choseBetterLotteryBundle ==1, "TRUE", "FALSE")) %>%
  ggplot(aes(probFractalDraw, mean_logRt, color=choseBetterLotteryBundle))+
  geom_point()+
  geom_errorbar(aes(ymin = mean_logRt - sem_logRt, ymax = mean_logRt + sem_logRt), width=.2)+
  theme(legend.position = "bottom")+
  labs(x="p(Fractal)", color="Chose Bundle with Better Lottery", y = "mean log RT")+
  scale_color_manual(values = cbbPalette[1:2])+
  facet_grid(conflictTrial~.)

Choice based on fractal

Are they faster when they choose the bundle with the better fractal? No systematic difference.

p = clean_beh_data %>%
  group_by(probFractalDraw, choseBetterFractalBundle) %>%
  summarise(.groups="keep",
            mean_logRt = mean(logRt),
            sem_logRt = sd(logRt)/sqrt(n())) %>%
  mutate(choseBetterFractalBundle = ifelse(choseBetterFractalBundle ==1, "TRUE", "FALSE")) %>%
  ggplot(aes(probFractalDraw, mean_logRt, color=choseBetterFractalBundle))+
  geom_point()+
  geom_errorbar(aes(ymin = mean_logRt - sem_logRt, ymax = mean_logRt + sem_logRt), width=.2)+
  theme(legend.position = "bottom")+
  labs(x="p(Fractal)", color="Chose Bundle with Better Fractal", y = "mean log RT")+
  scale_color_manual(values = cbbPalette[2:1])

# ggsave(file=paste0(fig_out_path, 'rt_by_correct_fractal.jpg'), p, height = 5, width=8, units="in")
p

RT patterns to model

Attribute relevance effects
- The more similar each attribute is in its relevance for reward the slower the choice.
- When only the fractals matter choice is categorically faster.

Value difference effects - Choice is faster for larger EV difference when lotteries matter more for reward.
- RT does NOT depend on fractal value difference, implying that when modeling RT this might not have to be taken into account.

Task learning effect - Subjects are on average faster the more trials they have done especially going from the first to the second run.

Choice effects - Subjects are slightly faster when they choose the bundle with the better lottery when the lottery matters more for the reward. This suggests less/no integration of additional attribute (fractals) when correctly choosing based on the relevant attribute. Trials where the lotteries mattered more for the reward but the better lottery bundle was not selected can be thought of as errors or choices in which the deliberation process was contaminated by the an irrelevant attribute (opposite of a speed-accuracy tradeoff).This effect is driven more by no conflict trials. There is no difference in RT for conflict trials depending on whether the bundle with the better lottery was chosen. Maybe the detection as a conflict draws attention/highlights the relevance of an attribute for a reward.
- A similar effect does not exist for choosing the bundle with the better fractal when fractals are more relevant for reward. This might support the idea that processing of fractals happens before the stimuli are presented (/before we begin measuring RT).